Migrate integration and provider HTTP types (PR 13)#626
Conversation
ChristianPavilonis
left a comment
There was a problem hiding this comment.
PR Review — Migrate integration and provider HTTP types (PR 13)
1 blocking finding, 5 non-blocking suggestions.
Blocking
🔧 Missing Content-Type forwarding for Didomi POST/PUT — see inline comment.
Non-blocking (inline)
- 🤔 Prebid still imports from
fastly::http— last remaininguse fastlyin the integration layer - ♻️ Duplicated body collection helpers (
collect_response_body/collect_body_bytes) - 🌱 Unbounded response body collection (matches pre-migration, but could use a size cap)
- ⛏ Dead
"X-"uppercase branch in custom header copy (http crate lowercases names) - ⛏ Duplicated
backend_name_for_urlhelper across 4 integrations
…/edgezero-pr13-integration-provider-type-migration
ChristianPavilonis
left a comment
There was a problem hiding this comment.
Summary
Solid migration of the integration and auction provider layers from Fastly-specific HTTP types to platform-agnostic http crate types. No blocking issues found. CI is passing.
Non-blocking
♻️ refactor
- Duplicated
body_as_readerhelper:proxy.rs:24andpublisher.rs:23define identicalbody_as_readerfunctions. Consider extracting into a shared location (e.g., aninto_reader()method onEdgeBody, or a helper inhttp_util.rs).
📝 note
- PR description file table incomplete:
proxy.rsandpublisher.rshave changes in this diff (thebody_as_readerextraction and test type alias cleanups) but are not listed in the PR body's change table.
📌 out of scope
- Orchestrator tests disabled pending mock
PlatformHttpClient: Several provider integration tests and timeout enforcement paths inorchestrator.rs:751-765are disabled pending a mock forPlatformHttpClient::select(). This is acknowledged in the PR, but the scope of untested paths has grown — a follow-up to add thin mock support forselect()would significantly improve coverage of the deadline enforcement logic.
aram356
left a comment
There was a problem hiding this comment.
Summary
The refactor is clean and the ensure_integration_backend / collect_body helpers extracted in the fix commits are a genuine improvement. The main risks are latent panics and missing POST size bounds that were introduced in the base migration commit. None are new in the fix commits, but this PR is the natural owner of the HTTP-type boundary, so they should land before merge.
Blocking
🔧 wrench
EdgeBody::into_bytes()panics on streaming bodies atauction/endpoints.rs:42— the inbound auction POST body is user-controlled.Body::into_bytes()panics onBody::Stream(_)(unit testinto_bytes_panics_for_streamasserts this). Today Fastly's adapter happens to returnBody::Once, so nothing panics — but that invariant is undocumented and the/auctionhandler is the first publicly-reachable crash if it ever changes. Fix: switch toBody::into_bytes_bounded(max).awaitwith a documented cap (e.g. 256KB).EdgeBody::into_bytes()panics across every migrated provider/proxy — same root cause, see inline comments onprebid.rs:1157,permutive.rs:151,lockr.rs:149,datadome.rs:301,aps.rs:554,adserver_mock.rs:373. Makeparse_responseasync and read bodies withinto_bytes_bounded.- Unbounded inbound POST body forwarding on the datadome / didomi / permutive / lockr proxies — only GTM enforces a cap. Promote the GTM size-bounded reader into
integrations/mod.rsascollect_body_boundedand apply it uniformly. Inline comments ondatadome.rs:367,didomi.rs:245,permutive.rs:211,lockr.rs:203. - Lockr
.expect()on user-controlledOriginheader — reachable worker panic atlockr.rs:266. - GTM stream-read errors mis-classified as
PayloadTooLarge— returns 413 for transport errors atgoogle_tag_manager.rs:313. Add aStreamReadvariant and map to 502.
Non-blocking
🤔 thinking
collect_bodyhas no size cap (integrations/mod.rs:101) — silently drains whole bodies to RAM. Same concern applies to the GTM response-rewrite path atgoogle_tag_manager.rs:486.- Orchestrator
select_result.readyerror path drops provider identity inrun_providers_parallel— whenreadyisErr, the provider identity is lost so noAuctionResponse::error(...)is pushed and the failing backend vanishes silently from the result set. Track a(backend_name, provider_index)pair alongsidefastly_pendingso failures are attributable. - Deadline enforcement relies on every provider honoring
backend_name(effective_timeout)— Phase 1 computesremaining_ms.min(provider.timeout_ms())correctly, but nothing guarantees each provider actually threads that timeout through to the backend config. Not a new regression, but the async migration makes it more load-bearing. Add a unit test that asserts the select loop returns beforedeadline + epsilon.
♻️ refactor
aps.rs/adserver_mock.rsstill buildBackendConfiginline — extend the newensure_integration_backendhelper to cover providers too.
🌱 seedling
- Make
parse_responseasync in the provider trait (auction/provider.rs:48) — needed anyway once theinto_bytes_boundedfix lands; zero cost under#[async_trait(?Send)].
📝 note
- Testlight stale
Content-Lengthtest is narrow (testlight.rs:425) — good unit-level coverage, but the twohandle()rebuild call sites would benefit from an end-to-end assertion.
👍 praise
- Clean helper extraction in
integrations/mod.rs— real dedup across six integrations. - Dropping the dead
|| name_str.starts_with("X-")branch inlockr.rs/permutive.rs—HeaderName::as_str()is always lowercase, so it was unreachable. - Adding
CONTENT_TYPEto didomi's forwarded headers — POST bodies can now be interpreted by upstream.
CI Status
- browser integration tests: PASS
- integration tests: PASS
- prepare integration artifacts: PASS
Conflicts resolved by keeping PR13's http-native integration layer throughout — all compat shim insertions from PR12 are superseded by this branch's completed integration type migration.
- Add collect_body_bounded helper with INTEGRATION_MAX_BODY_BYTES (256 KiB) cap to prevent unbounded memory use on streaming bodies - Replace all into_bytes() calls (panic on Body::Stream) with collect_body or collect_body_bounded across integrations and auction endpoint - Make AuctionProvider::parse_response async so implementations can safely drain response bodies without panicking on the Stream variant - Add .await to both parse_response call sites in the orchestrator - Cap inbound request bodies in lockr, permutive, datadome, and didomi proxy handlers using collect_body_bounded before forwarding upstream - Fix lockr Origin header: replace expect() with a warn-and-skip fallback so an invalid user-supplied origin cannot crash the edge worker - Add PayloadSizeError::StreamRead variant to google_tag_manager and return 502 (not 413) when a stream transport error occurs while reading the body - Remove extra blank line before closing brace in google_tag_manager impl block
aram356
left a comment
There was a problem hiding this comment.
Summary
Re-review of the EdgeZero PR13 integration/provider HTTP type migration after the latest "Address review findings" commit. The async parse_response migration and the new collect_body / collect_body_bounded helpers landed cleanly and resolved the prior into_bytes()-on-Stream panic class across all providers. Two blockers remain plus one CI/process question that affects whether the existing test plan can be trusted.
Blocking
🔧 wrench
-
cargo fmt --all -- --checkfails locally in 7+ files. The PR description's[x] cargo fmtcheckbox is incorrect — see the CI question below for why this wasn't caught. Affected files:crates/trusted-server-core/src/auction/endpoints.rs:39-47crates/trusted-server-core/src/auction/orchestrator.rs:12-14, 402-405crates/trusted-server-core/src/integrations/google_tag_manager.rs:16-25, 39-45crates/trusted-server-core/src/integrations/lockr.rs:282-285crates/trusted-server-core/src/integrations/permutive.rs:208-219, 278-281crates/trusted-server-core/src/integrations/prebid.rs:5-13, 1154-1162crates/trusted-server-core/src/integrations/testlight.rs:11-16
Fix: run
cargo fmt --all. -
Testlight POST body is unbounded (inline at
integrations/testlight.rs:181).
❓ question
-
CI gates aren't enforced for this PR.
.github/workflows/format.ymland.github/workflows/test.ymlonly fire onpull_request: branches: [main]. PR 626's base isfeature/edgezero-pr12-handler-layer-types, so on commit94ec9477only "Integration Tests" ran — fmt, clippy, andcargo test --workspacedid not. Combined with the fmt failure above, every stacked EdgeZero PR ships with broken formatting and an unverified test plan. Was this intentional, or should the workflow triggers be widened (or the gates duplicated under the integration-tests umbrella)? Until that's resolved, the PR description's CI checkboxes can't be relied on as evidence.Verified locally on this commit:
cargo test --workspacepasses (821 tests),cargo clippy --workspace --all-targets --all-features -- -D warningspasses,cargo build --release --target wasm32-wasip1passes,cargo fmt --all -- --checkfails.
Non-blocking
🤔 thinking
-
Unbounded response-body reads via
collect_body(inline atintegrations/mod.rs:93for stream-read error fidelity; cross-cutting concern listed here). The bounded variant exists for inbound bodies, but every upstream response site still drains an unbounded body to aVec<u8>:integrations/prebid.rs:1157— Prebid Server response (highest risk: third-party hop, large payloads)integrations/aps.rs:555— APS responseintegrations/adserver_mock.rs:375— Mediator responseintegrations/lockr.rs:150— Lockr SDK fetchintegrations/permutive.rs:152— Permutive SDK fetchintegrations/datadome.rs:302— DataDome SDK fetchintegrations/google_tag_manager.rs:491— GTM script
Provider responses are the highest risk — a misbehaving Prebid Server / mediator can ship a multi-GB body and OOM the worker. Recommend routing all of these through
collect_body_boundedwith a per-callsite cap, or removing the unbounded variant in favor of a single bounded helper.
♻️ refactor
-
Auction providers still construct
BackendConfiginline while integration proxies route throughensure_integration_backend. Reraising the prior-review concern with current locations:integrations/aps.rs:517-528integrations/adserver_mock.rs:338-348integrations/prebid.rs:1131-1135
The two paths differ on timeout (providers need a configurable per-request timeout; the proxy helper hardcodes 15s). Consider widening
ensure_integration_backendto accept a timeout and reusing it from the providers, or at least documenting the divergence.
CI Status
- fmt: FAIL locally (CI did not run for this commit — see ❓ question above)
- clippy: PASS locally (CI did not run)
- rust tests: PASS locally — 821 tests (CI did not run)
- wasm32-wasip1 build: PASS locally
- Integration Tests workflow: PASS (only workflow that ran on
94ec9477)
… compat conflicts
…EADME - Run cargo fmt --all (7+ files had formatting failures) - Cap testlight POST body with collect_body_bounded + INTEGRATION_MAX_BODY_BYTES - Use dedicated AUCTION_MAX_BODY_BYTES (256 KiB) for /auction instead of INTEGRATION_MAX_BODY_BYTES - Update auction/README.md: parse_response is now async, parallel execution via select() is live
…/edgezero-pr13-integration-provider-type-migration
aram356
left a comment
There was a problem hiding this comment.
Summary
Re-review of PR13 after the latest "Address PR review" commit (4b27087e) and the PR12 merge (a2036eb5). The async parse_response migration, the dedicated AUCTION_MAX_BODY_BYTES, the README updates, and the testlight body cap all landed cleanly. One blocker remains plus several non-blocking re-raises from prior reviews that haven't been addressed.
Local verification on this commit:
cargo test --workspace— PASS (952 tests, 0 failures)cargo clippy --workspace --all-targets --all-features -- -D warnings— PASScargo build --release --target wasm32-wasip1 -p trusted-server-adapter-fastly— PASScargo fmt --all -- --check— FAIL (4 PR-introduced sites + 11 inherited from PR12 base)- GitHub CI (integration tests workflow) — PASS
GitHub's format.yml / test.yml still only fire on PRs targeting main, so the fmt/clippy/test gates don't run for this PR's stack. The PR description's [x] cargo fmt / [x] cargo clippy / [x] cargo test checkboxes therefore can't be relied on as evidence — see the fmt blocker below.
Blocking
🔧 wrench
-
fmt failures still present in PR-modified files.
cargo fmt --all -- --checkreports diffs at:crates/trusted-server-adapter-fastly/src/main.rs:209and:236crates/trusted-server-core/src/integrations/prebid.rs:1057(the line at ~1066 is too long inside theif letblock)crates/trusted-server-core/src/integrations/registry.rs:673
These four sites did not exist on the PR12 base — PR13 introduced them. (11 additional fmt diffs are inherited from the PR12 base and are not this PR's responsibility, but the project-wide
cargo fmt --all -- --checkdoes fail.) Fix:cargo fmt --all.
Non-blocking
🤔 thinking
- Provider/integration response bodies still unbounded (re-raise; inline at
integrations/mod.rs:84). 7 callsites still drain upstream responses withcollect_bodyinstead of a bounded helper. Provider responses are highest risk (prebid.rs:1160,aps.rs:555,adserver_mock.rs:375). - Stream-read errors classified as
Integration(re-raise; inline atintegrations/mod.rs:93and the bounded variant at:137). Conflates transport failure with integration semantic error; both surface as 5xx-server with the same shape. GTM has a bespoke 502 path that the shared helper doesn't. - Orchestrator backend-name fallback can silently lose bids (inline at
auction/orchestrator.rs:337). Whenpending.backend_name()isNoneand the platform sets a differentPlatformResponse::backend_name, the lookup at line 392 fails and the bid is dropped with a"Received response from unknown backend"warning. Consider alog::warn!when the fallback fires. collect_body_boundedchecks size after reading the chunk (inline atintegrations/mod.rs:143). The effective bound is "≤ max + one_chunk", not strict ≤ max. Acceptable in practice (chunk sizes are small) but worth a comment on the helper.
♻️ refactor
- Auction providers still construct
BackendConfiginline (re-raise; inline atprebid.rs:1134, alsoaps.rs:517-528,adserver_mock.rs:338-348). Diverges fromensure_integration_backendbecause providers need a per-request timeout and the helper hardcodes 15s. AuctionContext.client_infois redundant withservices.client_info()(re-raise; inline atauction/types.rs:107).run_auctiontakesservicesas a separate arg even thoughcontext.servicesexists (inline atauction/orchestrator.rs:64). They must stay in sync; an inconsistent caller breaks the auction silently.- Inconsistent
services.client_infoaccess (inline atdidomi.rs:259). Two sites use thepub(crate)field (didomi.rs:259,auction/formats.rs:145); the rest use the methodservices.client_info(). Standardize on the method.
📝 note
- Lockr/Permutive/Didomi handlers contain unreachable PUT/PATCH branches (inline at
lockr.rs:202). Routes register only GET/POST. StubHttpClient::send_asyncdoes not capture request headers (inline atplatform/test_support.rs:297). Limits header-shape assertions for prebid/aps/adserver_mock tests.
🏕 camp site
testlight::Defaultimpls can be derived (inline attestlight.rs:271).
👍 praise
compat::shim eliminated from integration dispatch —IntegrationProxy::handleis nowhttp::Request<EdgeBody>end-to-end (main.rs:195,registry.rs:266). Removes a non-trivial conversion overhead per request and a class of header-mapping bugs.- Async
parse_responsemigration withPlatformResponseresolves the priorinto_bytes()-on-Body::Streampanic class for every provider (auction/provider.rs:38). The trait docs explicitly note why it's now async — exactly the right context for future implementors. - GTM 413 vs 502 distinction (
google_tag_manager.rs:511-535). Clean separation between client-error and transport-error; this is the pattern the sharedcollect_body*helpers should follow. - EC-ID header-injection guard (
registry.rs:673) plus regression test atregistry.rs:1406. DefensiveHeaderValue::from_strblocks\r/\n/\0injection on a user-influenced ID before it lands in either the request or the response.
CI Status
- fmt: FAIL locally (CI did not run for this commit because workflow triggers don't match the PR's base branch — see header above)
- clippy: PASS locally (CI did not run)
- rust tests: PASS locally (CI did not run; 952 tests)
- wasm32-wasip1 build: PASS locally
- Integration Tests workflow: PASS
aram356
left a comment
There was a problem hiding this comment.
Inline annotations for the prior REQUEST_CHANGES review
Adding the per-line comments now that I've filtered them against the PR diff (the previous submission was body-only because GitHub rejected the batch when 3 of 17 comment anchors fell outside the changed hunks). These reference the same findings as #626 (review) — no new findings, just inline anchoring.
The three findings that couldn't be anchored to a diff line are tagged below for reference:
- ♻️ refactor —
run_auctiontakesservicesseparately even thoughcontext.servicesexists. Anchored as a top-level note rather than inline becauseauction/orchestrator.rs:64(therun_auctionsignature) is not in the diff, but the dual-pass pattern is visible at the response-handling sites (e.g.orchestrator.rs:337). - 📝 note —
StubHttpClient::send_asyncdoes not capture request headers.crates/trusted-server-core/src/platform/test_support.rsis not in this PR's diff scope at all; the asymmetry betweensend(lines 266–280) andsend_async(lines 297–320) limits header-shape assertions for prebid/aps/adserver_mock tests. Worth a follow-up issue. - 🏕 camp site —
testlight::Defaultimpls can be derived.testlight.rs:271-285is outside the changed hunks; deriving#[derive(Default)]would replace both manual impls.
aram356
left a comment
There was a problem hiding this comment.
Inline annotations for the prior REQUEST_CHANGES review
Adding the per-line comments now that I've filtered them against the PR diff (the previous submission was body-only because GitHub rejected the batch when 3 of 17 comment anchors fell outside the changed hunks). These reference the same findings as #626 (review) — no new findings, just inline anchoring.
The three findings that couldn't be anchored to a diff line are tagged below for reference:
- ♻️ refactor —
run_auctiontakesservicesseparately even thoughcontext.servicesexists. Anchored as a top-level note rather than inline becauseauction/orchestrator.rs:64(therun_auctionsignature) is not in the diff, but the dual-pass pattern is visible at the response-handling sites (e.g.orchestrator.rs:337). - 📝 note —
StubHttpClient::send_asyncdoes not capture request headers.crates/trusted-server-core/src/platform/test_support.rsis not in this PR's diff scope at all; the asymmetry betweensend(lines 266–280) andsend_async(lines 297–320) limits header-shape assertions for prebid/aps/adserver_mock tests. Worth a follow-up issue. - 🏕 camp site —
testlight::Defaultimpls can be derived.testlight.rs:271-285is outside the changed hunks; deriving#[derive(Default)]would replace both manual impls.
…/edgezero-pr13-integration-provider-type-migration
aram356
left a comment
There was a problem hiding this comment.
Summary
Re-review of the EdgeZero PR13 integration/provider HTTP type migration after the latest commit (96949807). The async parse_response, collect_response_bounded, and dedicated AUCTION_MAX_BODY_BYTES constant landed cleanly and resolved several prior findings. Three blocking issues remain, all rooted in the same anti-pattern: change_context over collect_body_bounded discards the RequestTooLarge error variant, so oversized inbound bodies surface as 502 BAD_GATEWAY instead of 413 PAYLOAD_TOO_LARGE at every site except testlight. A test added in this PR (auction_rejects_oversized_body) already proves the bug.
CI status (verified locally on 96949807)
- fmt: PASS
- clippy: FAIL — 3
doc_markdownerrors atintegrations/mod.rs:172, 175 - rust tests: FAIL —
auction_rejects_oversized_bodypanics (asserts 413, gets 502) - wasm32-wasip1 release build: PASS
- JS tests: PASS — 282 tests
- Integration Tests workflow: PASS (only workflow that ran on this commit)
The PR description's [x] cargo clippy and [x] cargo test --workspace checkboxes are not corroborated by CI on this branch — .github/workflows/{format,test}.yml only fire on pull_request: branches: [main], so on commit 96949807 only the integration-tests workflow ran. Both gates fail when run locally.
Blocking
🔧 wrench
cargo clippy --workspace --all-targets --all-features -- -D warningsfails atintegrations/mod.rs:172, 175(3 doc-markdown errors).auction_rejects_oversized_bodyfails becauseauction/endpoints.rs:46change_contextsRequestTooLarge(→ 413) intoAuction(→ 502).- Same bug at every other
collect_body_boundedcaller except testlight:datadome.rs:362-364,didomi.rs:244-246,permutive.rs:213-217,lockr.rs:205-207. All produce 502 instead of 413 for oversized bodies, defeating the bounded-read security goal.testlight.rs:182-184is the only correct caller — its?-only pattern is the model.
Non-blocking
🤔 thinking
- Unbounded upstream-response read in testlight (
testlight.rs:214). Every other RTB integration migrated tocollect_response_bounded; testlight is the last unbounded site. - Predicted-backend-name fallback at
log::debug(orchestrator.rs:339-344). Failure mode is silent bid loss; recommendlog::warn!so operators can see the fallback firing.
♻️ refactor
- Three providers still inline
BackendConfig::from_url_with_first_byte_timeout(aps.rs:519-528,adserver_mock.rs:340-348,prebid.rs:1141-1145). Justification comments are reasonable; wideningensure_integration_backendwith afirst_byte_timeout: Durationparameter would let all four sites share one helper.
📝 note
- Testlight stale-
Content-Lengthtest is unit-only (testlight.rs:388-412). Extendinghandle_uses_platform_http_client_with_http_requestto inject a staleContent-Lengthon the stubbed upstream would close the integration-level gap.
Blocking fixes: - Fix doc_markdown clippy errors in integrations/mod.rs (BAD_GATEWAY, max_bytes, one_chunk wrapped in backticks) - Drop change_context() on collect_body_bounded() calls in auction, datadome, didomi, lockr, permutive — RequestTooLarge (413) was being rewritten to Integration/Auction (502); now propagates unchanged via ? so oversized bodies correctly surface as PAYLOAD_TOO_LARGE - Remove debug_assert! in compat::to_fastly_response (already fixed in PR 12, missed in PR 13 merge) Non-blocking fixes: - Migrate testlight upstream response read from unbounded collect_body to collect_response_bounded(UPSTREAM_RTB_MAX_RESPONSE_BYTES) matching all other RTB integrations; remove now-unused collect_body function - Upgrade orchestrator predicted-backend-name fallback log from debug to warn so operators see silent bid-loss fallback in production
Conflict resolution: keep PR13 http-native integration dispatch (no compat round-trip) and add HandlerOutcome::Buffered wrapping from PR12. Migrate sourcepoint.rs from fastly SDK types to edgezero http types, completing the integration layer HTTP type migration started in PR13. All sourcepoint helper methods and tests updated to use http::Request, http::Response, and HeaderValue instead of fastly::Request/Response.
aram356
left a comment
There was a problem hiding this comment.
Summary
Re-review of the EdgeZero PR13 integration/provider HTTP type migration after the May 12 fix commits and the May 12 merge that imported sourcepoint into PR13.
The prior-review blockers from Apr 14 and Apr 20 (into_bytes()-on-Stream panics, unbounded inbound POST forwarding, lockr Origin expect, GTM 413/502 classification, unbounded testlight POST) are all resolved cleanly — collect_body_bounded / collect_response_bounded are used uniformly across prebid / aps / adserver_mock / testlight / lockr / permutive / datadome / gtm, with thoughtful RTB (2 MiB) vs SDK (16 MiB) sizing. New tests on the auction endpoint, GTM payload limits, and testlight stale Content-Length directly close prior findings.
Two new blockers were introduced by the May 12 sourcepoint merge, plus one outstanding CI/process question.
Blocking
🔧 wrench
-
Sourcepoint silently drops streaming JS response bodies — see inline at
crates/trusted-server-core/src/integrations/sourcepoint.rs:802. Behavioral regression vs.main. Fix: usecollect_response_bounded. -
cargo fmt --all -- --checkfails on 8 hunks insourcepoint.rs— lines 315, 417, 680, 703, 801, 1431, 1472, 1552. Same blocker as the Apr 20 review, reintroduced when sourcepoint was added to PR13 in the May 12 merge. The May 12 "Apply cargo fmt formatting" commit ran before the sourcepoint merge. Fix:cargo fmt --all.
❓ question
-
CI gates still aren't enforced for this PR.
.github/workflows/format.ymlandtest.ymlstill trigger onpull_request: branches: [main]. PR 626's base isfeature/edgezero-pr12-handler-layer-types, so only "Integration Tests" ran on commit28205a00— fmt, clippy, andcargo test --workspacedid not. Combined with the fmt failure above, the PR description's CI checkboxes can't be relied on. Same question as the Apr 20 review; no workflow change has been made. Is the plan to (a) widen workflow triggers to include the stackedfeature/edgezero-*base branches, (b) require stacked branches to retarget main before merge, or (c) fold format/clippy/test into the integration-tests workflow that does run?Verified locally on commit
28205a00:cargo clippy --workspace --all-targets --all-features -- -D warningspasses;cargo test --workspacepasses (954 unit tests in core);cargo fmt --all -- --checkfails.
Non-blocking
🤔 thinking
-
Orchestrator drops failing provider identity on
select()error —crates/trusted-server-core/src/auction/orchestrator.rs:436-440. Whenselect_result.readyisErr, noAuctionResponse::error(provider, …)is pushed. Thebackend_to_providermap still holds entries for the in-flight providers, but the failedPlatformPendingRequestis consumed and there's no way to look the provider up. The failing backend vanishes from the result set silently — only alog::warn!with the error message, no provider name. Theresponsesarray under-counts launched providers. Same concern as the Apr 14 review.Suggested fix: preserve the launch-time mapping in a separate
Vec<(backend_name, provider_name)>indexed by the position of eachPlatformPendingRequest, so theErrarm can produce a properAuctionResponse::error(provider_name, …). -
Deadline drop and
effective_timeout == 0paths are untested —crates/trusted-server-core/src/auction/orchestrator.rs:446-453(deadline check) and:294(zero-timeout skip). The in-tree TODO at:740-754explicitly acknowledges this. Both paths are load-bearing for the async migration's correctness.Suggested fix: introduce a thin abstraction around
PlatformHttpClient::select()that takes pre-baked futures, plus a unit test that asserts the loop returns withindeadline + εwhen one provider is artificially slow. -
GTM
accumulated_textbuffer can persist stale state across requests —crates/trusted-server-core/src/integrations/google_tag_manager.rs:194,629,641. TheArc<GoogleTagManagerIntegration>is shared across requests, and the accumulator is drained only whenis_last_in_text_node. If an upstream HTML stream errors mid-script (orlol_htmlaborts the document), residual fragments remain in the buffer and get concatenated into the next request's first matching script — corrupting the rewrite output for an unrelated document. Pre-existing (present inmain); flagged here because PR13 is the natural owner of the integration layer. Could be a 📌 follow-up issue if you'd prefer to keep PR scope tight.
♻️ refactor
- Auction providers still construct
BackendConfiginline —crates/trusted-server-core/src/integrations/aps.rs:519-529,crates/trusted-server-core/src/integrations/adserver_mock.rs:340-350,crates/trusted-server-core/src/integrations/prebid.rs:1521-1525. Providers diverge from proxies on this single call because they need a per-request timeout, whileensure_integration_backendinintegrations/mod.rshardcodes 15 s. Widening the helper to accept anOption<Duration>for the timeout would let three call sites reuse it. Same refactor opportunity flagged in the prior review; non-blocking.
🌱 seedling
- A streaming-aware rewrite helper would close the sourcepoint class of bugs. Sourcepoint's
EdgeBody::Stream(_)regression is symptomatic of every "rewrite body" path needing to convert Stream → Once. If something likerewrite_response_body(body, max, integration, |s| rewrite_fn(s))lived inintegrations/mod.rs, integrations couldn't accidentally skip the Stream variant.
CI Status
- fmt: FAIL locally (CI did not run for this commit — see ❓ question)
- clippy: PASS locally (CI did not run)
- rust tests: PASS locally — 954 unit tests in
trusted-server-core(CI did not run) - Integration Tests workflow: PASS (only workflow triggered on
28205a00)
Blocking fixes: - Use collect_response_bounded for sourcepoint JS response bodies, replacing the EdgeBody::Stream branch that silently returned an empty body; streaming responses now drain up to the 5 MiB cap and surface oversized bodies as Integration/502, matching all other integrations - Run cargo fmt to fix 8 formatting hunks in sourcepoint.rs introduced by the May 12 merge Non-blocking: - Orchestrator now identifies the failing provider on select() Err by diffing backend_to_provider keys against the new remaining list; logs with provider name and pushes AuctionResponse::error so the response array counts correctly Refactor: - Add first_byte_timeout: Option<Duration> to ensure_integration_backend; proxy call sites pass None (keeps 15 s default); auction providers (aps, adserver_mock, prebid) switch from BackendConfig::from_url_with_first_byte_timeout to ensure_integration_backend with the auction-scoped timeout, going through the platform abstraction consistently
|
Resolved all findings from the round-1 review in commit f551544. Blocking — fixed
Non-blocking — addressed
Refactor — addressed
Deferred (reviewer explicitly gave outs)
CI: |
Conflict in auction/endpoints.rs: keep PR13's collect_body_bounded (async, streaming-safe, bounds before materializing) over PR12's enforce_max_body_size + into_bytes() (sync, allocates full body before checking the cap).
aram356
left a comment
There was a problem hiding this comment.
Summary
Re-review of the EdgeZero PR13 integration/provider HTTP type migration after the May 28 fix commits (f5515440, 2b17d4c9). The two prior blockers — sourcepoint streaming-body drop and the cargo fmt failures — are genuinely fixed, as are the lockr Origin panic, the 413/502 classification, and the testlight stale Content-Length. The first_byte_timeout refactor is correct and preserves proxy behavior.
However, the fix for the previously non-blocking "orchestrator drops failing provider identity" finding introduces a new production correctness bug (detailed inline), and a separate stale Content-Length forwarding issue in datadome needs to be addressed. Verdict: REQUEST_CHANGES.
Blocking
🔧 wrench
- Orchestrator mis-identifies the failed provider on
select()error in production — inline atauction/orchestrator.rs:436. The diff-against-remaining-names logic relies onbackend_name()surviving inremaining, which the Fastly adapter strips toNone(works under the test mock, fails in production: blames an arbitrary provider and can drop that provider's real later response). - datadome forwards a client-supplied stale
Content-Lengthonto a re-framed body — inline atintegrations/datadome.rs:387.
❓ question
- CI gates still don't run for this PR.
format.yml/test.ymltrigger onpull_request: branches: [main], and this PR's base is the stackedfeature/edgezero-pr12-handler-layer-types, so only "Integration Tests" ran in CI — fmt, clippy, andcargo test --workspacedid not. I verified all three pass locally on2b17d4c9(fmt clean, clippy clean with-D warnings, 982 core unit tests pass), but the PR's CI checkboxes can't be relied on as a gate. Is the plan to (a) widen workflow triggers to the stackedfeature/edgezero-*bases, (b) retarget tomainbefore merge, or (c) fold fmt/clippy/test into the integration-tests workflow that does run? (Same question as prior reviews; no workflow change has been made.)
Non-blocking
🤔 thinking
- Deadline-drop and
effective_timeout == 0paths remain untested —auction/orchestrator.rs(TODO at ~761). Load-bearing for the async migration's correctness; the in-tree TODO acknowledges it. Aligning the mockselect()to stripbackend_namefromremainingexactly like the Fastly adapter would also have surfaced the wrench finding above — worth doing alongside an Err-path test with ≥2 in-flight providers.
🌱 seedling
- A streaming-aware rewrite helper would close the sourcepoint class of bugs. A shared
rewrite_response_body(body, max, integration, |s| ...)inintegrations/mod.rswould structurally prevent integrations from accidentally skipping theBody::Streamvariant.
👍 praise
- All four prior blockers are genuinely fixed: sourcepoint Stream-body drop now uses
collect_response_bounded+ aContent-Lengthguard (sourcepoint.rs:805-811);cargo fmtpasses; testlight drops the staleContent-Lengthwith a dedicated regression test (testlight.rs:121,394); GTM 413-vs-502 classification is correct. - The
first_byte_timeoutrefactor is correct —None.unwrap_or_else(|| 15s)exactly preserves proxy behavior, all three RTB providers thread the auction timeout, and the predicted vs. ensured backend names agree. - Bounded body reads are uniform and correctly classified: inbound oversize → 413 propagated via a bare
?(nochange_contextrewriting it to 502), upstream oversize → 502; the lockrOriginexpect()is replaced with a warn-and-skip fallback. main.rsdrops twocompatround-trips per proxy request — a genuine cleanup.
CI Status (verified locally on head 2b17d4c9)
- fmt: PASS
- clippy: PASS (
-D warnings) - rust tests: PASS (982 core unit tests)
- Integration Tests (CI): PASS
- fmt / clippy /
cargo test --workspacein CI: did not run (see ❓ question)
Migrates all conflicting files to use http::Request<EdgeBody>/http::Response<EdgeBody> (PR13 types) while incorporating PR12 additions: EC lifecycle in handle_proxy, ProxyDispatchInput struct, EID gating in auction endpoints, and platform-agnostic parallel bidding via PlatformHttpClient::select. Key resolutions: - orchestrator.rs: rewrote parallel bidding loop to use PlatformHttpClient::select; fixed mediator wait to use mediator_context.services - registry.rs: handle_proxy uses http types with EC generate_if_needed lifecycle - prebid.rs: parse_response uses PlatformResponse (async); removed 6 stale fastly::Response-based tests superseded by platform-agnostic implementation - provider.rs: removed auto-merged parse_response_with_context (fastly::Response signature, no callers after orchestrator migration) - main.rs: ProxyDispatchInput call wired with http-typed fields, no compat shims
- Fix orchestrator select() Err path: add failed_backend_name to PlatformSelectResult and populate it from fastly::SendError::backend_name() in the Fastly adapter; orchestrator now uses this field directly instead of the broken diff-against-remaining logic (which always failed on Fastly because remaining entries have no backend names) - Add push_select_error() to StubHttpClient and strip backend names from remaining entries in select() to match Fastly production behavior - Add select_error_is_attributed_to_correct_provider test: two stub providers, one fails via injected select error, assert correct attribution - Remove CONTENT_LENGTH from datadome headers_to_copy (body is re-materialized so client Content-Length is wrong for the outbound request) - Replace _request_info ignored param in prebid to_openrtb with request_info and remove the duplicate internal RequestInfo::from_request call - Replace from_utf8_lossy with strict from_utf8 + fallback in google_tag_manager - Replace .expect() on HeaderValue::from_str with if let Ok in adserver_mock
aram356
left a comment
There was a problem hiding this comment.
Summary
Migration of the integration/provider layers from Fastly request/response types to platform-agnostic http types (async request_bids, PlatformPendingRequest/PlatformResponse, threaded RuntimeServices). Reviewed against the stacked base feature/edgezero-pr12-handler-layer-types. The change is clean and well-hardened: no Fastly/compat leakage remains in trusted-server-core, response/request body reads are centrally bounded with distinct RTB/SDK/request caps, stale Content-Length is dropped on every body-rewrite path, and select() errors are now correctly attributed to a provider via the new failed_backend_name. No blocking issues found.
Non-blocking
🤔 thinking
- Timed-out providers vanish from the response: deadline-dropped requests produce no
AuctionResponse(unlike launch failures) — consider emitting a timeout error per dropped backend (orchestrator.rs:513). - Inconsistent error metadata: transport failures push a bare error while parse failures attach
error_type/message(orchestrator.rs:493 vs 471).
📌 out of scope
backend_to_providerkeyed only by backend name: providers sharing a backend collide; one response is dropped/misattributed. Pre-existing in base; worth a follow-up guard (orchestrator.rs:379).- Timeout-enforcement paths untested: acknowledged TODO — deadline drop, mediator skip, and
effective_timeout == 0skip have no coverage;StubHttpClientcould now exercise some (orchestrator.rs:950).
⛏ nitpick
- Repeated
total_time_ms: 0placeholder across threeOrchestrationResultconstructions (orchestrator.rs:172/244/263).
CI Status
- browser integration tests: PASS
- integration tests: PASS
- prepare integration artifacts: PASS
| { | ||
| let response_time_ms = start_time.elapsed().as_millis() as u64; | ||
| log::warn!("Provider '{}' request failed: {:?}", provider_name, e); | ||
| responses.push(AuctionResponse::error(provider_name, response_time_ms)); |
There was a problem hiding this comment.
🤔 thinking — Inconsistent error metadata between failure modes.
This transport-failure path pushes a bare AuctionResponse::error(...), while the parse-failure path (line 471) attaches error_type/message diagnostic metadata via provider_error_response. The public /auction response shape therefore differs depending on how a provider failed.
Suggestion: route this through a shared helper with a dedicated classifier (e.g. ERROR_TYPE_TRANSPORT) so transport failures carry the same metadata envelope. Non-blocking.
Related (pre-existing, outside this diff): deadline-dropped providers at ~line 513 produce no AuctionResponse at all — consider draining the leftover backend_to_provider entries into timeout error responses for observability.
| ); | ||
| backend_name.clone() | ||
| }); | ||
| backend_to_provider.insert( |
There was a problem hiding this comment.
📌 out of scope — backend_to_provider is keyed solely by backend name, so two enabled providers that resolve to the same backend (same server_url + same effective timeout) collide: this insert overwrites the first mapping while both pending requests are still launched. When the responses arrive, only one maps; the other is logged "unknown backend, ignoring" and dropped/misattributed.
This is pre-existing (the same keying is in the base branch), not introduced by this PR — but worth a follow-up guard or a collision warn! at insert time. Non-blocking.
| @@ -838,18 +948,19 @@ mod tests { | |||
| } | |||
|
|
|||
| // TODO: Re-enable provider integration tests after implementing mock support | |||
There was a problem hiding this comment.
📌 out of scope — The timeout-enforcement paths called out in this TODO remain untested: the deadline-drop in the select() loop, the mediator skip on exhausted budget, and the provider skip when effective_timeout == 0. The new StubHttpClient introduced in this PR could now exercise at least some of these without real backends. Suggest tracking via a follow-up issue so the gap doesn't get lost. Non-blocking.
…yer-types' into feature/edgezero-pr13-integration-provider-type-migration # Conflicts: # crates/trusted-server-adapter-fastly/src/platform.rs # crates/trusted-server-core/src/auction/orchestrator.rs # crates/trusted-server-core/src/auction/provider.rs # crates/trusted-server-core/src/integrations/didomi.rs # crates/trusted-server-core/src/integrations/registry.rs # crates/trusted-server-core/src/platform/test_support.rs
Provider transport failures (select() errors) pushed a bare AuctionResponse::error with no metadata, while parse and launch failures attach error_type/message. Route transport failures through a shared helper with a new ERROR_TYPE_TRANSPORT classifier and a static, upstream-safe message, so the public /auction response shape no longer depends on how a provider failed. Add unit coverage for the new helper.
80a521a
into
feature/edgezero-pr12-handler-layer-types
…ation and provider HTTP types (PR 13) (#626) * Rename crates to trusted-server-core and trusted-server-adapter-fastly Rename crates/common → crates/trusted-server-core and crates/fastly → crates/trusted-server-adapter-fastly following the EdgeZero naming convention. Add EdgeZero workspace dependencies pinned to rev 170b74b. Update all references across docs, CI workflows, scripts, agent files, and configuration. * Add platform abstraction layer with traits and RuntimeServices Introduces trusted-server-core::platform with PlatformConfigStore, PlatformSecretStore, PlatformKvStore, PlatformBackend, PlatformHttpClient, and PlatformGeo traits alongside ClientInfo, PlatformError, and RuntimeServices. Wires the Fastly adapter implementations and threads RuntimeServices into route_request. Moves GeoInfo to platform/types as platform-neutral data and adds geo_from_fastly for field mapping. * Address platform layer review feedback - Defer KV store opening: replace early error return with a local UnavailableKvStore fallback so routes that do not need synthetic ID access succeed when the KV store is missing or temporarily unavailable - Use ConfigStore::try_open + try_get and SecretStore::try_get throughout FastlyPlatformConfigStore and FastlyPlatformSecretStore to honour the Result contract instead of panicking on open/lookup failure - Encapsulate RuntimeServices service fields as pub(crate) with public getter methods (config_store, secret_store, backend, http_client, geo) and a pub new() constructor; adapter updated to use new() - Reference #487 in FastlyPlatformHttpClient stub (PR 6 implements it) - Remove unused KvPage re-export from platform/mod.rs - Use super::KvHandle shorthand in RuntimeServices::kv_handle() * Reject host strings containing control characters in BackendConfig * Fix clippy error * Validate scheme and host for control characters in BackendConfig * Address review findings on platform abstraction layer * Address review findings on platform abstraction layer * Add config store read path and storage module split - Split fastly_storage.rs into storage/{config_store,secret_store,api_client,mod}.rs - Add PlatformConfigStore read path via FastlyPlatformConfigStore::get using ConfigStore::try_open/try_get - Add PlatformError::NotImplemented variant; stub write methods on FastlyPlatformConfigStore and FastlyPlatformSecretStore - Add StoreName/StoreId newtypes with From<String>, From<&str>, AsRef<str> - Add UnavailableKvStore to core platform module - Add RuntimeServicesBuilder replacing 7-arg constructor - Migrate get_active_jwks and handle_trusted_server_discovery to use &RuntimeServices - Update call sites in signing.rs, rotation.rs, main.rs - Add success-path test for handle_trusted_server_discovery using StubJwksConfigStore - Fix test_parse_cookies_to_jar_empty typo (was emtpy) * Harden legacy config-store reads and align Fastly adapter stubs * Address storage review feedback * Resolved github-advanced-security bot problems * Address PR review feedback on platform abstraction layer - Make StoreName and StoreId inner fields private; From/AsRef provide all needed construction and access - Add #[deprecated] to GeoInfo::from_request with #[allow(deprecated)] at the three legacy call sites to track migration progress - Enumerate the six platform traits in the platform module doc comment - Extract backend_config_from_spec helper to remove duplicate BackendConfig construction in predict_name and ensure - Replace .into_iter().collect() with .to_vec() on secret plaintext bytes - Remove unused bytes dependency from trusted-server-adapter-fastly - Add comment on SecretStore::open clarifying it already returns Result (unlike ConfigStore::open which panics) * Add PR 4 design spec for secret store trait (read-only) * Clarify test scope and deferred branches in PR 4 spec * Add implementation plan for PR 4 secret store trait * Add test for get_secret_bytes open-failure path * Add NotImplemented tests for FastlyPlatformSecretStore write stubs * Inline StoreId binding and add section comment in write-stub tests * Remove plan * Add PR 6 design spec for backend and HTTP client traits * Address spec review findings on PR 6 design * Implement PlatformHttpClient and thread RuntimeServices through proxy layer - Add PlatformHttpClient trait with send(), send_async(), and select() methods - Add PlatformBackend trait with predict_name() and ensure() methods - Add PlatformResponse wrapper around EdgeZero HTTP responses - Add PlatformPendingRequest and PlatformSelectResult for auction fan-out - Thread RuntimeServices through IntegrationProxy::handle(), IntegrationRegistry::handle_proxy(), and all first-party proxy endpoints so handlers can reach the HTTP client - Add StubHttpClient and StubBackend test stubs with build_services_with_http_client helper - Add proxy_request_calls_platform_http_client_send integration test - Fix proxy_with_redirects to stay within 7-arg clippy limit via ProxyRequestHeaders struct - Document Body::Stream limitation in edge_request_to_fastly with warning log - Document intentional duplication of platform_response_to_fastly across proxy and orchestrator - Remove spec file (promoted to plan + implementation) * Address pr review findings * Resolve pr review findings * Add PR7 design spec for geo lookup + client info extract-once Documents the call site migration plan: five Fastly SDK extraction points in trusted-server-core replaced by RuntimeServices::client_info reads, following Phase 1 injection pattern from the EdgeZero migration design. * Fix spec review issues in PR7 design doc - Correct erroneous claim about generate_synthetic_id being called twice via DeviceInfo; it is called once (line 91 for fresh_id), DeviceInfo.ip is a separate req.get_client_ip_addr() call fixed independently - Add before/after snippet for handle_publisher_request call site in main.rs - Add noop_services import instruction for http_util.rs test module - Clarify _services rename (drop underscore, not add new param) in didomi.rs - Clarify nextjs #[allow(deprecated)] annotations are out of scope (different function) * Update PR7 spec to address all five agent review findings - Change RequestInfo::from_request signature to &ClientInfo (not &RuntimeServices) so prebid can call it with context.client_info - Scope SDK-call acceptance criteria to active non-deprecated code only - List all six AuctionContext construction sites including two production sites in orchestrator.rs and three test helpers in orchestrator/prebid - Add explicit warn-and-continue pattern for publisher.rs geo lookup - Correct testing table: formats.rs and endpoints.rs have no test modules; add orchestrator.rs and prebid.rs test helper update rows * Add PR7 implementation plan and address plan review findings Plan covers 6 tasks in compilation-safe order: AuctionContext struct change first, then from_request signature, then synthetic.rs cascade, then publisher geo, then didomi. Includes two new copy_headers unit tests (Some/None). Spec fixes: clarify injection pattern exceptions for &ClientInfo and Option<IpAddr>; reword acceptance criterion to reflect that provider-layer reads flow through AuctionContext.client_info. * Fix three plan review findings and two open questions - Finding 1 (High): Add missing publisher.rs test call site at line ~695 for get_or_generate_synthetic_id — was omitted from Task 3 Step 6 - Finding 2 (Medium): Remove crate::geo::GeoInfo import from endpoints.rs rather than replacing it — type is not used by name after the change, keeping any import fails clippy -D warnings - Finding 3 (Low): Replace interactive git add -p in Task 6 with explicit file staging instruction - Open Q1: Add Task 2 step to update stale handle_publisher_request signature in auction/README.md - Open Q2: Add Task 2 step to update from_request doc comment to reflect ClientInfo-based TLS detection instead of Fastly SDK calls * Broaden two low-severity doc cleanup steps in PR7 plan - Step 7: cover all four stale Fastly-SDK-specific locations in http_util.rs (SPOOFABLE_FORWARDED_HEADERS doc, RequestInfo struct doc, from_request doc, detect_request_scheme doc) - Step 8: replace the whole routing snippet in auction/README.md, not just the one handle_publisher_request line — handle_auction and integration_registry.handle_proxy are also stale in that snippet * Fix two remaining low findings in PR7 plan - Add missing Location 2 (RequestInfo.scheme field doc, line ~67) to Step 7; renumber subsequent locations 3-5 - Replace &runtime_services with runtime_services in Step 5 and README snippet — runtime_services is already &RuntimeServices in route_request * Fix count drift in Step 7: four → five locations * Add client_info field to AuctionContext and fix all construction sites * Change RequestInfo::from_request to take &ClientInfo, thread services into handle_publisher_request * Add Task 2 follow-up coverage and README route fixes * Add services param to generate_synthetic_id, remove Fastly IP/geo calls in formats and endpoints * Revert premature publisher geo change from Task 3 * Replace deprecated GeoInfo::from_request in publisher.rs with services.geo().lookup() * Remove Fastly IP extraction from Didomi copy_headers, use ClientInfo instead * Move IpAddr import to test module level in didomi.rs * Apply rustfmt formatting to didomi.rs, publisher.rs, and synthetic.rs Fix multi-line function call style in didomi.rs, line-break wrapping in publisher.rs test, and import ordering in synthetic.rs test module. * Add test coverage for generate_synthetic_id with concrete client IP Adds noop_services_with_client_ip helper to test_support and a new test that verifies the client_ip path through generate_synthetic_id by asserting the HMAC differs when the IP changes. * Align geo lookup warn log format with codebase convention ({e} not {e:?}) * Apply Prettier formatting to PR7 plan and spec docs * Document content rewriting as platform-agnostic in platform module * Document html_processor as platform-agnostic * Document streaming_processor as platform-agnostic * Fix unresolved doc link: replace EdgeRequest with edgezero_core::http::Request * Add plan for content rewriting * Add plan for PR9: wire signing to store primitives * Add build_services_with_config_and_secret to test_support * Add FastlyManagementApiClient to adapter * Implement FastlyPlatformConfigStore and FastlyPlatformSecretStore write methods via management API Replace FastlyApiClient with FastlyManagementApiClient in the put/delete methods of FastlyPlatformConfigStore and the create/delete methods of FastlyPlatformSecretStore. Remove the now-unused FastlyApiClient import. * Migrate KeyRotationManager from FastlyApiClient to RuntimeServices store primitives * Migrate signing.rs from FastlyConfigStore/FastlySecretStore to RuntimeServices * Delete storage/api_client.rs from core; remove FastlyApiClient * Fix formatting after CI gate check * Add services to AuctionContext; remove deprecated from_config shim Thread RuntimeServices into AuctionContext so auction providers can access platform stores directly. Update PrebidAuctionProvider to use RequestSigner::from_services(context.services) instead of the now- removed from_config() shim. All construction sites and test helpers updated accordingly. This satisfies the final acceptance criterion of #490: no FastlyConfigStore/FastlySecretStore construction remains in the request_signing/ modules. * Fix prettier formatting in PR9 plan document * Add PR 10 logging initialization design * Add PR 10 logging initialization plan * Fix PR 10 logging plan to avoid per-log allocation * Extract Fastly logging initialization into adapter module * Wire Fastly main.rs to adapter-local logging module * Remove log-fastly from core dependencies * Format Fastly logging module declaration * format plan docs * Address PR findings * Restore idiomatic fern logging and improve target label extraction - Reverted gratuitous _message rename and record.args() usage in logging.rs, returning to the idiomatic message parameter inside the fern format closure. - Refactored target_label to use .rsplit_once("::") rather than .split("::").last(). This provides a more explicit and robust way to extract the final module segment. - Expanded target_label test coverage to explicitly test edge cases such as inputs without :: separators, empty strings, and inputs with trailing ::. * Migrate utility layer to HTTP types * Migrate handler layer to HTTP types * Address PR review findings * Address review findings * Address review findings * Resolve review findings * Resolve PR review findings * Address review findings * Removed unused import * Fix rotate/delete atomicity, HTTP verb, idempotent deletes, and weak tests * Resolve PR review feedback on logging module * Address review findings * Resolve PR review findings * Address round-3 review findings - Revert proxy.rs merge artifact: restore per-request allowed_domains at both redirect_is_permitted call sites; remove dead_code allow and stale comment — integration proxies defaulting to &[] get open mode again as documented - Drop unused trusted-server-js dep from adapter Cargo.toml - Fix check_response: gate body read behind error branch so 2xx paths do not buffer and discard the response body - Remove self-referential SECRET_UPSERT_METHOD test - Reorder write-cost doc so outbound HTTPS round-trip leads; handle-open caching noted as negligible - Refactor make_request to take fastly::http::Method; drop string match and unreachable arm; remove SECRET_UPSERT_METHOD const - Add SigningStoreIds named struct in endpoints.rs; update both call sites to destructure by name * Address PR review: add body-size caps and remove orchestrator duplication - Add VERIFY_MAX_BODY_BYTES (4096) cap to handle_verify_signature - Add AUCTION_MAX_BODY_BYTES (65536) cap to handle_auction - Add ADMIN_MAX_BODY_BYTES (4096) cap to handle_rotate_key and handle_deactivate_key - Delete platform_response_to_fastly from orchestrator; both call sites now use crate::compat::to_fastly_response directly - Add sign_rejects_oversized_body and rebuild_rejects_oversized_body regression tests asserting 413 on oversized POST bodies * Resolve PR review findings * Resolve PR review findings * Resolve PR review findings * Use append_header in place of set_header * fix lint * Route Fastly cookie calls through compat bridge after PR10 merge cookies.rs set_ec_cookie/expire_ec_cookie now take Response<EdgeBody> to satisfy the migration_guards invariant. registry.rs and publisher.rs call the Fastly-typed equivalents in compat instead. Remove synthetic.rs entry from migration_guards (file deleted in PR10). * Remove unused Logger import * Resolve PR review findings * Fix fmt lint * Rename synthetic_id_cookie_value_is_safe → ec_cookie_value_is_safe * Resolve PR review findings * Resolve PR 624 review findings Blocking fixes: - Remove debug_assert! in compat::to_fastly_response that contradicted the documented silent-drop fallback and caused to_fastly_response_with_streaming_body_produces_empty_body to panic in debug builds - Restore streaming dispatch for PublisherResponse::Stream, which was regressed by the PR 11 compat shim. Introduce HandlerOutcome enum so route_request can signal streaming intent back to the adapter. main() handles HandlerOutcome::Streaming via to_fastly_response_skeleton + stream_to_client + stream_publisher_body directly, avoiding the Vec<u8> buffer that delayed TTFB and scaled WASM memory with body size. Non-blocking fixes: - Add to_fastly_response_skeleton for headers-only fastly response conversion - Add geo-401 comment: skip geo lookup for unauthenticated callers - Add audited-callers comment on EdgeBody::Stream arms in compat - Add O(N) / PR 15 comment on bytes.to_vec() calls - Deduplicate auction/publisher status extraction in route_tests via outcome_status helper * Call StreamingBody::finish() to properly terminate chunked response StreamingBody in fastly 0.11.12 has no Drop impl — dropping it without calling finish() leaves the chunked HTTP stream with no terminal chunk, causing clients to receive "unexpected EOF during chunk size line". Match on stream_publisher_body result and call finish() in both the success and error paths so the response is always properly terminated. * Apply cargo fmt formatting * Resolve PR 624 round-6 review findings Blocking fixes: - Restore drop(streaming_body) on error path so client sees EOF/truncated response instead of a silently completed partial response (cd4c621 regression) - Add HandlerOutcome::AuthChallenge variant to distinguish our enforce_basic_auth 401s from origin-forwarded 401s; gate geo lookup on AuthChallenge only so origin-forwarded 401s still carry geo headers Non-blocking fixes: - Fix stream_publisher_body doc: remove incorrect async claim; function is sync - Clarify to_fastly_request_ref doc: non-empty body is a caller error - Add enforce_max_body_size helper in http_util; replace 6 duplicated body-size cap blocks across proxy, auction, and request_signing endpoints - Add test for to_fastly_response_skeleton covering status + header round-trip - Restore "bytes" unit in enforce_max_body_size error message for log clarity - Widen enforce_max_body_size what param from &'static str to &str * Bound publisher response body size and document interim buffering Add MAX_PLATFORM_RESPONSE_BODY_BYTES (10 MiB) cap in fastly_response_to_platform to prevent heap exhaustion from large origin responses. Content-Length pre-check avoids the WASM heap copy entirely for declared-size responses; post-materialization check covers chunked responses without Content-Length. Document the interim buffering regression in PublisherResponse::Stream and PassThrough doc comments, noting that both variants now hold a fully materialised body until the platform HTTP client gains streaming response support in PR 15. Resolves PR 624 round-7 review findings. * Add edge_cookie module, test helpers, and integration handle signatures Complete merge resolution of remaining unstaged files: - lib.rs: expose edge_cookie as pub(crate) module (used by proxy.rs) - platform/test_support.rs: add recorded_request_headers() on StubHttpClient and build_services_with_http_client() factory for tests with custom HTTP clients - integrations/{datadome,didomi,lockr,permutive,prebid,sourcepoint}: add _services: &RuntimeServices parameter to handle() following main branch IntegrationProxy trait change * Fix remaining code review findings from merge Address medium-priority findings surfaced during review of the main merge: - test_support.rs: record request headers in send_async (was only recorded in send), so recorded_request_headers() is consistent across sync and async HTTP test paths - test_support.rs: document StubBackend vs NoopBackend distinction on build_services_with_http_client so callers understand the tradeoff - endpoints.rs: 413 PAYLOAD_TOO_LARGE response now includes Content-Type and a client-facing message instead of an empty body with a server-internal error string - kv.rs: derive Clone for KvIdentityGraph (trivial String wrapper) - main.rs: build KvIdentityGraph once and clone for finalize_kv_graph, eliminating the double call to maybe_identity_graph * Fix cargo fmt lint * Migrate auction format tests to edgezero http types after merge Merge 6e191e9 combined the edgezero http::Request/Response migration with new auction tests from main that still used the Fastly SDK API, and added the RequestTooLarge error variant. Update the auction format tests to build Request<EdgeBody> via the http builder, read response bodies through EdgeBody::into_bytes, pass the new services and geo arguments to convert_tsjs_to_auction_request, and use http accessors. Cover the new RequestTooLarge variant in the error status-code guard and mapping test. * Migrate integration and provider HTTP types (PR 13) (#626) * Migrate integration and provider HTTP types * Resolve PR findings * Resolve PR findings * Address review findings: safe body reads and bounded inbound forwarding - Add collect_body_bounded helper with INTEGRATION_MAX_BODY_BYTES (256 KiB) cap to prevent unbounded memory use on streaming bodies - Replace all into_bytes() calls (panic on Body::Stream) with collect_body or collect_body_bounded across integrations and auction endpoint - Make AuctionProvider::parse_response async so implementations can safely drain response bodies without panicking on the Stream variant - Add .await to both parse_response call sites in the orchestrator - Cap inbound request bodies in lockr, permutive, datadome, and didomi proxy handlers using collect_body_bounded before forwarding upstream - Fix lockr Origin header: replace expect() with a warn-and-skip fallback so an invalid user-supplied origin cannot crash the edge worker - Add PayloadSizeError::StreamRead variant to google_tag_manager and return 502 (not 413) when a stream transport error occurs while reading the body - Remove extra blank line before closing brace in google_tag_manager impl block * Address PR review: fmt, testlight body cap, auction constant, stale README - Run cargo fmt --all (7+ files had formatting failures) - Cap testlight POST body with collect_body_bounded + INTEGRATION_MAX_BODY_BYTES - Use dedicated AUCTION_MAX_BODY_BYTES (256 KiB) for /auction instead of INTEGRATION_MAX_BODY_BYTES - Update auction/README.md: parse_response is now async, parallel execution via select() is live * Resolve PR review findings * Resolve PR 626 review findings Blocking fixes: - Fix doc_markdown clippy errors in integrations/mod.rs (BAD_GATEWAY, max_bytes, one_chunk wrapped in backticks) - Drop change_context() on collect_body_bounded() calls in auction, datadome, didomi, lockr, permutive — RequestTooLarge (413) was being rewritten to Integration/Auction (502); now propagates unchanged via ? so oversized bodies correctly surface as PAYLOAD_TOO_LARGE - Remove debug_assert! in compat::to_fastly_response (already fixed in PR 12, missed in PR 13 merge) Non-blocking fixes: - Migrate testlight upstream response read from unbounded collect_body to collect_response_bounded(UPSTREAM_RTB_MAX_RESPONSE_BYTES) matching all other RTB integrations; remove now-unused collect_body function - Upgrade orchestrator predicted-backend-name fallback log from debug to warn so operators see silent bid-loss fallback in production * Apply cargo fmt formatting * Resolve PR 626 round-1 review findings Blocking fixes: - Use collect_response_bounded for sourcepoint JS response bodies, replacing the EdgeBody::Stream branch that silently returned an empty body; streaming responses now drain up to the 5 MiB cap and surface oversized bodies as Integration/502, matching all other integrations - Run cargo fmt to fix 8 formatting hunks in sourcepoint.rs introduced by the May 12 merge Non-blocking: - Orchestrator now identifies the failing provider on select() Err by diffing backend_to_provider keys against the new remaining list; logs with provider name and pushes AuctionResponse::error so the response array counts correctly Refactor: - Add first_byte_timeout: Option<Duration> to ensure_integration_backend; proxy call sites pass None (keeps 15 s default); auction providers (aps, adserver_mock, prebid) switch from BackendConfig::from_url_with_first_byte_timeout to ensure_integration_backend with the auction-scoped timeout, going through the platform abstraction consistently * Resolve PR 626 round-2 review findings - Fix orchestrator select() Err path: add failed_backend_name to PlatformSelectResult and populate it from fastly::SendError::backend_name() in the Fastly adapter; orchestrator now uses this field directly instead of the broken diff-against-remaining logic (which always failed on Fastly because remaining entries have no backend names) - Add push_select_error() to StubHttpClient and strip backend names from remaining entries in select() to match Fastly production behavior - Add select_error_is_attributed_to_correct_provider test: two stub providers, one fails via injected select error, assert correct attribution - Remove CONTENT_LENGTH from datadome headers_to_copy (body is re-materialized so client Content-Length is wrong for the outbound request) - Replace _request_info ignored param in prebid to_openrtb with request_info and remove the duplicate internal RequestInfo::from_request call - Replace from_utf8_lossy with strict from_utf8 + fallback in google_tag_manager - Replace .expect() on HeaderValue::from_str with if let Ok in adserver_mock * Give auction transport failures a consistent error envelope Provider transport failures (select() errors) pushed a bare AuctionResponse::error with no metadata, while parse and launch failures attach error_type/message. Route transport failures through a shared helper with a new ERROR_TYPE_TRANSPORT classifier and a static, upstream-safe message, so the public /auction response shape no longer depends on how a provider failed. Add unit coverage for the new helper.
Summary
RuntimeServicesHTTP client primitives with asyncrequest_bids,PlatformPendingRequest, andPlatformResponsewhile preserving orchestrator deadline handling.Changes
crates/trusted-server-adapter-fastly/src/main.rsRuntimeServicesinto the PR13 entrypoint.crates/trusted-server-core/src/auction/README.mdPlatformPendingRequest,PlatformResponse, and the platform HTTP client flow.crates/trusted-server-core/src/auction/endpoints.rsRuntimeServicesinto the auction entrypoint context.crates/trusted-server-core/src/auction/orchestrator.rsPlatformPendingRequest/select, and update the remaining PR13 migration comments.crates/trusted-server-core/src/auction/provider.rsAuctionProviderto asyncrequest_bidswithPlatformPendingRequest/PlatformResponseand refresh the trait docs.crates/trusted-server-core/src/auction/types.rsRuntimeServicestoAuctionContext.crates/trusted-server-core/src/integrations/adserver_mock.rscrates/trusted-server-core/src/integrations/aps.rscrates/trusted-server-core/src/integrations/datadome.rscrates/trusted-server-core/src/integrations/didomi.rscrates/trusted-server-core/src/integrations/google_tag_manager.rscrates/trusted-server-core/src/integrations/gpt.rscrates/trusted-server-core/src/integrations/lockr.rscrates/trusted-server-core/src/integrations/permutive.rscrates/trusted-server-core/src/integrations/prebid.rsPlatformResponseresults.crates/trusted-server-core/src/integrations/registry.rsIntegrationProxy/routing types tohttptypes and threadRuntimeServicesthrough proxy dispatch.crates/trusted-server-core/src/integrations/testlight.rsContent-Lengthregression test, and drop stale length headers when rebuilding JSON responses.Closes
Closes #494
Test plan
cargo test --workspacecargo clippy --workspace --all-targets --all-features -- -D warningscargo fmt --all -- --checkcd crates/js/lib && npx vitest runcd crates/js/lib && npm run formatcd docs && npm run formatcargo build --package trusted-server-adapter-fastly --release --target wasm32-wasip1fastly compute servecargo doc --no-deps --all-features(passes with pre-existing unrelated rustdoc warnings outside this PR's scope)Checklist
unwrap()in production code — useexpect("should ...")println!